On the conditions used to prove oracle results for the Lasso
نویسنده
چکیده
Oracle inequalities and variable selection properties for the Lasso in linear models have been established under a variety of different assumptions on the design matrix. We show in this paper how the different conditions and concepts relate to each other. The restricted eigenvalue condition (Bickel et al., 2009) or the slightly weaker compatibility condition (van de Geer, 2007) are sufficient for oracle results. We argue that both these conditions allow for a fairly general class of design matrices. Hence, optimality of the Lasso for prediction and estimation holds for more general situations than what it appears from coherence (Bunea et al., 2007b,c) or restricted isometry (Candès and Tao, 2005) assumptions.
منابع مشابه
Some theoretical results on the grouped variables Lasso
We consider the linear regression model with Gaussian error. We estimate the unknown parameters by a procedure inspired from the Group Lasso estimator introduced in [21]. We show that this estimator satisfies a sparsity oracle inequality, i.e., a bound in terms of the number of non-zero components of the oracle vector. We prove that this bound is better, in some cases, than the one achieved by ...
متن کاملLASSO, Iterative Feature Selection and the Correlation Selector: Oracle Inequalities and Numerical Performances
Abstract: We propose a general family of algorithms for regression estimation with quadratic loss, on the basis of geometrical considerations. These algorithms are able to select relevant functions into a large dictionary. We prove that a lot of methods that have already been studied for this task (LASSO, Dantzig selector, Iterative Feature Selection, among others) belong to our family, and exh...
متن کاملOracle inequalities for cross-validation type procedures
Abstract We prove oracle inequalities for three different type of adaptation procedures inspired by cross-validation and aggregation. These procedures are then applied to the construction of Lasso estimators and aggregation with exponential weights with data-driven regularization and temperature parameters, respectively. We also prove oracle inequalities for the crossvalidation procedure itself...
متن کاملAdaptive Elastic Net: An Improvement of Elastic Net
Lasso proved to be an extremely successful technique for simultaneous estimation and variable selection. However lasso has two major drawbacks. First, it does not capture any grouping effect and secondly in some situations lasso solutions are inconsistent. To overcome inconsistency recently adaptive lasso was proposed where adaptive weights are used for penalizing different coefficients. Adapti...
متن کاملThe Adaptive Lasso and Its Oracle Properties
The lasso is a popular technique for simultaneous estimation and variable selection. Lasso variable selection has been shown to be consistent under certain conditions. In this work we derive a necessary condition for the lasso variable selection to be consistent. Consequently, there exist certain scenarios where the lasso is inconsistent for variable selection. We then propose a new version of ...
متن کامل